71 research outputs found
Neural computation, social networks, and topological spectra
AbstractThis paper emphasizes some intriguing links between neural computation on graphical domains and social networks, like those used in nowadays search engines to score the page authority. It is pointed out that the introduction of web domains creates a unified mathematical framework for these computational schemes. It is shown that one of the major limitations of currently used connectionist models, namely their scarce ability to capture the topological features of patterns, can be effectively faced by computing the node rank according to social-based computation, like Google's PageRank. The main contribution of the paper is the introduction of a novel graph spectral notion, which can be naturally used for the graph isomorphism problem. In particular, a class of graphs is introduced for which the problem is proven to be polynomial. It is also pointed out that the derived spectral representations can be nicely combined with learning, thus opening the doors to many applications typically faced within the framework of neural computation
Constraint-Based Visual Generation
In the last few years the systematic adoption of deep learning to visual
generation has produced impressive results that, amongst others, definitely
benefit from the massive exploration of convolutional architectures. In this
paper, we propose a general approach to visual generation that combines
learning capabilities with logic descriptions of the target to be generated.
The process of generation is regarded as a constrained satisfaction problem,
where the constraints describe a set of properties that characterize the
target. Interestingly, the constraints can also involve logic variables, while
all of them are converted into real-valued functions by means of the t-norm
theory. We use deep architectures to model the involved variables, and propose
a computational scheme where the learning process carries out a satisfaction of
the constraints. We propose some examples in which the theory can naturally be
used, including the modeling of GAN and auto-encoders, and report promising
results in problems with the generation of handwritten characters and face
transformations
Multitask Kernel-based Learning with Logic Constraints
This paper presents a general framework to integrate prior knowledge in the
form of logic constraints among a set of task functions into kernel machines.
The logic propositions provide a partial representation of the environment, in
which the learner operates, that is exploited by the learning algorithm
together with the information available in the supervised examples. In
particular, we consider a multi-task learning scheme, where multiple unary
predicates on the feature space are to be learned by kernel machines and a
higher level abstract representation consists of logic clauses on these
predicates, known to hold for any input. A general approach is presented to
convert the logic clauses into a continuous implementation, that processes the
outputs computed by the kernel-based predicates. The learning task is
formulated as a primal optimization problem of a loss function that combines a
term measuring the fitting of the supervised examples, a regularization term,
and a penalty term that enforces the constraints on both supervised and
unsupervised examples. The proposed semi-supervised learning framework is
particularly suited for learning in high dimensionality feature spaces, where
the supervised training examples tend to be sparse and generalization
difficult. Unlike for standard kernel machines, the cost function to optimize
is not generally guaranteed to be convex. However, the experimental results
show that it is still possible to find good solutions using a two stage
learning schema, in which first the supervised examples are learned until
convergence and then the logic constraints are forced. Some promising
experimental results on artificial multi-task learning tasks are reported,
showing how the classification accuracy can be effectively improved by
exploiting the a priori rules and the unsupervised examples.Comment: The 19th European Conference on Artificial Intelligence (ECAI 2010
Multitask Kernel-based Learning with First-Order Logic Constraints
In this paper we propose a general framework to integrate supervised and
unsupervised examples with background knowledge expressed by a collection of
first-order logic clauses into kernel machines. In particular, we consider a
multi-task learning scheme where multiple predicates defined on a set of
objects are to be jointly learned from examples, enforcing a set of FOL
constraints on the admissible configurations of their values. The predicates
are defined on the feature spaces, in which the input objects are represented,
and can be either known a priori or approximated by an appropriate kernel-based
learner. A general approach is presented to convert the FOL clauses into a
continuous implementation that can deal with the outputs computed by the
kernel-based predicates. The learning problem is formulated as a
semi-supervised task that requires the optimization in the primal of a loss
function that combines a fitting loss measure on the supervised examples, a
regularization term, and a penalty term that enforces the constraints on both
the supervised and unsupervised examples. Unfortunately, the penalty term is
not convex and it can hinder the optimization process. However, it is possible
to avoid poor solutions by using a two stage learning schema, in which the
supervised examples are learned first and then the constraints are enforced.Comment: The 20th International Conference on Inductive Logic Programming (ILP
2010). Florence, Italy. June 27-30 201
Relational Neural Machines
Deep learning has been shown to achieve impressive results in several tasks
where a large amount of training data is available. However, deep learning
solely focuses on the accuracy of the predictions, neglecting the reasoning
process leading to a decision, which is a major issue in life-critical
applications. Probabilistic logic reasoning allows to exploit both statistical
regularities and specific domain expertise to perform reasoning under
uncertainty, but its scalability and brittle integration with the layers
processing the sensory data have greatly limited its applications. For these
reasons, combining deep architectures and probabilistic logic reasoning is a
fundamental goal towards the development of intelligent agents operating in
complex environments. This paper presents Relational Neural Machines, a novel
framework allowing to jointly train the parameters of the learners and of a
First--Order Logic based reasoner. A Relational Neural Machine is able to
recover both classical learning from supervised data in case of pure
sub-symbolic learning, and Markov Logic Networks in case of pure symbolic
reasoning, while allowing to jointly train and perform inference in hybrid
learning tasks. Proper algorithmic solutions are devised to make learning and
inference tractable in large-scale problems. The experiments show promising
results in different relational tasks
T-Norms Driven Loss Functions for Machine Learning
Neural-symbolic approaches have recently gained popularity to inject prior
knowledge into a learner without requiring it to induce this knowledge from
data. These approaches can potentially learn competitive solutions with a
significant reduction of the amount of supervised data. A large class of
neural-symbolic approaches is based on First-Order Logic to represent prior
knowledge, relaxed to a differentiable form using fuzzy logic. This paper shows
that the loss function expressing these neural-symbolic learning tasks can be
unambiguously determined given the selection of a t-norm generator. When
restricted to supervised learning, the presented theoretical apparatus provides
a clean justification to the popular cross-entropy loss, which has been shown
to provide faster convergence and to reduce the vanishing gradient problem in
very deep structures. However, the proposed learning formulation extends the
advantages of the cross-entropy loss to the general knowledge that can be
represented by a neural-symbolic method. Therefore, the methodology allows the
development of a novel class of loss functions, which are shown in the
experimental results to lead to faster convergence rates than the approaches
previously proposed in the literature
Enhancing Embedding Representations of Biomedical Data using Logic Knowledge
Knowledge Graph Embeddings (KGE) have become a quite popular class of models
specifically devised to deal with ontologies and graph structure data, as they
can implicitly encode statistical dependencies between entities and relations
in a latent space. KGE techniques are particularly effective for the biomedical
domain, where it is quite common to deal with large knowledge graphs underlying
complex interactions between biological and chemical objects. Recently in the
literature, the PharmKG dataset has been proposed as one of the most
challenging knowledge graph biomedical benchmark, with hundreds of thousands of
relational facts between genes, diseases and chemicals. Despite KGEs can scale
to very large relational domains, they generally fail at representing more
complex relational dependencies between facts, like logic rules, which may be
fundamental in complex experimental settings. In this paper, we exploit logic
rules to enhance the embedding representations of KGEs on the PharmKG dataset.
To this end, we adopt Relational Reasoning Network (R2N), a recently proposed
neural-symbolic approach showing promising results on knowledge graph
completion tasks. An R2N uses the available logic rules to build a neural
architecture that reasons over KGE latent representations. In the experiments,
we show that our approach is able to significantly improve the current
state-of-the-art on the PharmKG dataset. Finally, we provide an ablation study
to experimentally compare the effect of alternative sets of rules according to
different selection criteria and varying the number of considered rules
Contrastive Losses and Solution Caching for Predict-and-Optimize
Many decision-making processes involve solving a combinatorial optimization
problem with uncertain input that can be estimated from historic data.
Recently, problems in this class have been successfully addressed via
end-to-end learning approaches, which rely on solving one optimization problem
for each training instance at every epoch. In this context, we provide two
distinct contributions. First, we use a Noise Contrastive approach to motivate
a family of surrogate loss functions, based on viewing non-optimal solutions as
negative examples. Second, we address a major bottleneck of all
predict-and-optimize approaches, i.e. the need to frequently recompute optimal
solutions at training time. This is done via a solver-agnostic solution caching
scheme, and by replacing optimization calls with a lookup in the solution
cache. The method is formally based on an inner approximation of the feasible
space and, combined with a cache lookup strategy, provides a controllable
trade-off between training time and accuracy of the loss approximation. We
empirically show that even a very slow growth rate is enough to match the
quality of state-of-the-art methods, at a fraction of the computational cost.Comment: Accepted at IJCAI202
- …